Reliable and valid assessment of clinical bronchoscopy performance.

BACKGROUND There have been several attempts to systematically assess performance in bronchoscopy. Earlier validation studies have used bronchoscopy simulators, not real-life performance in patients. OBJECTIVES The aim of this study was to explore the reliability and validity of an assessment tool aimed for the use in a clinical setting. METHODS Five junior residents, 5 senior residents and 9 consultants performed 3 bronchoscopies each. All 57 bronchoscopies were video-recorded and assessed blindly and independently by two bronchoscopy experts using the new assessment tool. RESULTS The interrater reliability was high, with Cronbach's α = 0.86. Assessment of 3 bronchoscopies by a single rater had a generalizability coefficient of 0.84. The correlation between experience and performance was good (Pearson correlation = 0.76). There were significant differences between the groups for all aspects of the assessment, but post hoc tests showed different discriminative abilities. CONCLUSIONS This new tool for assessing clinical bronchoscopy performance has a high interrater reliability. One rater assessing performance of 3 bronchoscopies ensures sufficient reliability. The assessment tool demonstrated sufficient construct validity.